3D display apparatus and method for processing image using the same

ABSTRACT

A three-dimensional (3D) display apparatus is provided, including an image input device which receives an image and depth information, a multi-view image generator which generates a multi-view foreground image having depth information which is less than a preset depth value, and a multi-view rear ground image having depth information which is equal to or greater than the preset depth value, using the received image and depth information, a multi-view image renderer which performs rendering by arranging the multi-view foreground image according to a first arrangement pattern and the multi-view rear ground image according to a second arrangement pattern, and a display which outputs the rendered multi-view image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication No. 61/619,491, filed on Apr. 3, 2012, in the United StatesPatent and Trademark Office, and Korean Patent Application No.10-2012-0130308, filed on Nov. 16, 2012, in the Korean IntellectualProperty Office, the disclosures of which are incorporated herein byreference in their entirety.

BACKGROUND

1. Field

Apparatuses and methods consistent with the exemplary embodiments relateto a 3D display apparatus and a method for processing an image using thesame, and more particularly, to a 3D display apparatus which reduces adead zone which may be generated according to a viewing position of theviewer, and a method for processing an image using the same.

2. Description of the Related Art

Recently, an effort to develop a three-dimensional (3D) displayapparatus has been accelerated for viewing with cubic effect.Accordingly, 3D images which were able to be watched mainly in thetheater can be watched at home using a general display apparatus such asa television.

3D display apparatuses may be divided into a glass type system and aglasses-free system according to whether glasses are used for watching a3D image.

An example of a glass type system is a shutter glass display apparatus.In the shutter glass scheme, left-eye and right-eye images are outputalternately and left and right shutter glasses of 3D glasses worn by theviewer are opened or closed alternately in conjunction with the outputof left-eye and right-eye images so that the viewer can feel a cubiceffect.

A glasses-free system is also referred to as an autostereoscopic system.A glasses-free 3D display apparatus displays optically separatedmulti-view images and transmits light corresponding to images ofdifferent views to the viewer's left and right eyes using parallaxbarriers or lenticular lenses so that the viewer can feel a cubiceffect.

FIG. 1 illustrates the display of multi-view images of a generalglasses-free 3D display apparatus.

With reference to FIG. 1, multi-view images are reproduced by renderingoptically separated multi-view images from the 1^(st) view to the 9^(th)view in a way that the 1^(st) view image is positioned in the 1^(st)view position and the 9^(th) view image is positioned in the 9^(th) viewposition. If the viewer is positioned between the 1^(st) view and the9^(th) view, the viewer can watch a 3D image without the need of glassesand feel motion parallax according to the change of a viewing position.However, a general 3D display apparatus arranges and displays imagessequentially from the 1^(st) view to the 9^(th) view so that a dead zonemay be generated according to the viewing position.

A dead zone indicates a position where a viewing position of the viewerchanges from the N^(th) view to the 1^(st) view. In the dead zone, theviewer watches at the same time images of two views which are separatedfar away so that serious crosstalk may occur. Consequently, the viewercannot watch the 3D image. As shown in FIG. 1, the position where the9^(th) view image and the 1^(st) view image are viewed at the same timeis a dead zone.

FIG. 2 illustrates an arrangement pattern of multi-view images of ageneral glasses-free 3D display apparatus.

With reference to FIG. 2, multi-view images are arranged sequentiallyaccording to each optical view as illustrated in FIG. 1. Accordingly, aradical difference of image views, i.e. a dead zone, occurs in positionsof the 9^(th) view and the 1^(st) view.

In the dead zone, a 3D image cannot be viewed normally due to crosstalk.

SUMMARY

Exemplary embodiments of the present invention overcome the abovedisadvantages and other disadvantages not described above. Also, thepresent invention is not required to overcome the disadvantagesdescribed above, and an exemplary embodiment of the present inventionmay not overcome any of the problems described above.

The exemplary embodiments provide a 3D display apparatus and an imageprocessing method thereof, which allow the viewer to watch a 3D imagecomfortably at any position using a multi-view image processing methodto reduce dead zones.

According to an aspect of an exemplary embodiment, a three-dimensional(3D) display apparatus includes an image input device which receives animage and depth information, a multi-view image generator whichgenerates a multi-view foreground image having depth information whichis less than a preset depth value, and a multi-view rear ground imagehaving depth information which is equal to or higher than the presetdepth value, using the received image and depth information, amulti-view image renderer which performs rendering by arranging themulti-view foreground image according to a first arrangement pattern andthe multi-view rear ground image according to a second arrangementpattern, and a display which outputs the rendered multi-view image.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views, if “N” is an odd number (2K−1, K is a naturalnumber), the first arrangement pattern may be a repeat of a patternwhere 1^(st) view to K^(th) view are arranged sequentially and thenK−1^(th) view to 1^(st) view are arranged in reverse order, or if “N” isan even number (2K, K is a natural number), the first arrangementpattern may be a repeat of a pattern where 1^(st) view to K+1^(th) vieware arranged sequentially and then K^(th) view to 2^(nd) view arearranged in reverse order, and the second arrangement pattern may be apattern obtained by shifting the first arrangement pattern by apredetermined number of views and thereby having a phase difference.

The multi-view foreground image and the multi-view rear ground imageeach may have 9 views, the first arrangement pattern may be a patternwhere 1^(st), 2^(nd), 3^(rd), 4^(th), 5^(th), 4^(th), 3^(rd), 2^(nd),and 1^(st) view foreground images are arranged repeatedly, and thesecond arrangement pattern may be a pattern where 2^(nd), 3^(rd),4^(th), 5^(th), 4^(th), 3^(rd), 2^(nd), 1^(st), and 2^(nd) view rearground images is arranged repeatedly.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views, the first arrangement pattern may be a repeatof a pattern where odd views of 1^(st) to N^(th) views are arrangedsequentially and then even views of N^(th) to 1^(st) views are arrangedin reverse order, and the second arrangement pattern may be a patternobtained by shifting the first arrangement pattern by a predeterminednumber of views and thereby having a phase difference.

The multi-view foreground image and the multi-view rear ground imageeach may have 9 views, the first arrangement pattern may be a patternwhere 1^(st), 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th), 4^(th),and 2^(nd) view foreground images are arranged repeatedly, and thesecond arrangement pattern may be a pattern where 3^(rd), 5^(th),7^(th), 9^(th), 8^(th), 6^(th), 4^(th), 2^(nd), and 1^(st) viewforeground images are arranged repeatedly.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views, the first arrangement pattern may be a repeatof a pattern where odd views of 1^(st) to N^(th) views are arrangedsequentially and then even views of N^(th) to 1^(st) views are arrangedin reverse order, and if N is an odd number, the second arrangementpattern may be a repeat of a pattern where 1^(st) view is arrangedfirst, even views of 2^(nd) to N^(th) views are arranged sequentiallyand then odd views of N^(th) to 2^(nd) views are arranged in reverseorder, or if N is an even number, the second arrangement pattern may bea repeat of a pattern where even views of 1^(st) to N^(th) views arearranged sequentially and then odd views of N^(th) to 1^(st) views arearranged in reverse order.

The multi-view foreground image and the multi-view rear ground imageeach may have 9 views, the first arrangement pattern may be a patternwhere 1^(st), 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th), 4^(th),and 2^(nd) view foreground images are arranged repeatedly, and thesecond arrangement pattern may be a pattern where 1^(st), 2^(nd),4^(th), 6^(th), 8^(th), 9^(th), 7^(th), 5^(th), and 3^(rd) view rearground images are arranged repeatedly.

According to another aspect of an exemplary embodiment, an imageprocessing method of a 3D display apparatus is provided, the methodincluding receiving an image and depth information of the image,generating a multi-view foreground image having depth information whichis less than a preset depth value, and a multi-view rear ground imagehaving depth information which is equal to or higher than the presetdepth value, based on the received image and depth information,performing rendering by arranging the multi-view foreground imageaccording to a first arrangement pattern and the multi-view rear groundimage according to a second arrangement pattern, and outputting therendered multi-view image.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views. If “N” is an odd number (2K−1, K is a naturalnumber), the first arrangement pattern may be a repeat of a patternwhere 1^(st) view to K^(th) view are arranged sequentially and thenK−1^(th) view to 1^(st) view are arranged in reverse order, or if “N” isan even number (2K, K is a natural number), the first arrangementpattern may be a repeat of a pattern where 1^(st) view to K+1^(th) vieware arranged sequentially and then K^(th) view to 2^(nd) view arearranged in reverse order. The second arrangement pattern may be apattern obtained by shifting the first arrangement pattern by apredetermined number of views and thereby having a phase difference.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views. The first arrangement pattern may be a repeatof a pattern where odd views of 1^(st) to N^(th) views are arrangedsequentially and then even views of N^(th) to 1^(st) views are arrangedin reverse order, and the second arrangement pattern may be a patternobtained by shifting the first arrangement pattern by a predeterminednumber of views and thereby having a phase difference.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views. The first arrangement pattern may be a repeatof a pattern where odd views of 1^(st) to N^(th) views are arrangedsequentially and then even views of N^(th) to 1^(st) views are arrangedin reverse order. If N is an odd number, the second arrangement patternmay be a repeat of a pattern where the 1^(st) view is arranged first,even views of 2^(nd) to N^(th) views are arranged sequentially and thenodd views of the N^(th) to 2^(nd) views are arranged in reverse order.If N is an even number, the second arrangement pattern may be a repeatof a pattern where even views of 1^(st) to N^(th) views are arrangedsequentially and then odd views of N^(th) to 1^(st) views are arrangedin reverse order.

According to another aspect of an exemplary embodiment, a non-transitorycomputer readable medium which includes a program to execute an imageprocessing method of a 3D display apparatus is provided wherein theimage processing method includes receiving an image and depthinformation of the image, generating a multi-view foreground imagehaving depth information which is less than a preset depth value, and amulti-view rear ground image having depth information which is equal toor higher than the preset depth value, based on the received image andthe received depth information of the image, performing rendering byarranging the multi-view foreground image according to a firstarrangement pattern and the multi-view rear ground image according to asecond arrangement pattern, and outputting the rendered multi-viewimage.

Additional and/or other aspects of the exemplary embodiment will be setforth in part in the description which follows and, in part, will beobvious from the description, or may be learned by practice of theexemplary embodiments.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The above and/or other aspects will be more apparent by describingcertain exemplary embodiments with reference to the accompanyingdrawings, in which:

FIG. 1 illustrates display of multi-view images of a generalglasses-free 3D display apparatus;

FIG. 2 illustrates rendering of multi-view images of a generalglasses-free 3D display apparatus;

FIG. 3 is a block diagram illustrating a configuration of a 3D displayapparatus according to an exemplary embodiment;

FIG. 4 illustrates operation of a multi-view image generator accordingto an exemplary embodiment;

FIGS. 5 and 6 illustrate an arrangement pattern of multi-view imagesaccording to the first exemplary embodiment;

FIGS. 7 and 8 illustrate an arrangement pattern of multi-view imagesaccording to the second exemplary embodiment;

FIGS. 9 and 10 illustrate an arrangement pattern of multi-view imagesaccording to the third exemplary embodiment;

FIG. 11 illustrates operation of a multi-view image renderer accordingto an exemplary embodiment; and

FIG. 12 is a flow chart illustrating a method for processing an imageusing a 3D display apparatus according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Certain exemplary embodiments will now be described in greater detailwith reference to the accompanying drawings.

In the following description, same drawing reference numerals are usedfor the same elements even in different drawings. The matters defined inthe description, such as detailed construction and elements, areprovided to assist in a comprehensive understanding of the exemplaryembodiments. Thus, it is apparent that the exemplary embodiments can becarried out without those specifically defined matters. Also, well-knownfunctions or constructions are not described in detail since they wouldobscure the exemplary embodiments with unnecessary detail.

FIG. 3 is a block diagram illustrating a configuration of athree-dimensional (3D) display apparatus according to an exemplaryembodiment. A 3D display apparatus is an apparatus which displayscontent in a 3D manner, thereby allowing the viewer to feel a cubiceffect. Diverse devices such as televisions (TVs), monitors, personalcomputers (PCs), mobile phones, laptop computers, tablet PCs, electronicpicture frames, electronic books and personal digital assistants (PDAs)may be implemented with 3D display apparatuses. A 3D display apparatusaccording to the present exemplary embodiment may be implemented in aglasses-free 3D display scheme.

With reference to FIG. 3, the 3D display apparatus 100 may include animage input device 110, a multi-view image generator 120, a multi-viewimage renderer 130, and a display 140.

The image input device 110 receives an image and depth information ofthe image. More specifically, the image input device 110 may receive animage and depth information of the image from diverse external devicessuch as an external storage medium, a broadcasting station, a web serverand the like.

The input image may be one of a single-view image, a stereoscopic image,and a multi-view image. The single-view image is an image photographedby a general photographing device. The stereoscopic image is a 3D videoimage expressed only in a left and right image, which are cubic imagestaken by a stereoscopic photographing device. In general, a stereoscopicphotographing device is a photographing device having two lenses, whichare used to photograph a cubic image. The multi-view image is a 3D videoimage which provides the viewer with diverse views from multipledirections by geometrically correcting images taken by one or morephotographing devices and spatially composing the images.

The image input device 110 receives depth information of an image.

The depth information is a value of depth given to each pixel of theimage. For example, depth information of 8 bits may have values rangingfrom 0 to 255. In general, depth information may be obtained in apassive way by using the 2-dimensional feature of an image such asstereo matching or in an active way by using equipment such as a depthcamera. Depth information may be a depth map.

In general, as the distance of an image gets closer, a depth value getssmaller, and as the distance of the image gets further away from theviewer, a depth value gets larger, or vice versa. However, for the sakeof convenience, the exemplary embodiments will describe the example whena distance of an image gets closer, a depth value gets smaller.

The multi-view image generator 120 generates, based on the input imageand depth information of the image, a multi-view foreground image havingdepth information smaller than a preset depth value, and a multi-viewrear ground image having depth information equal to or larger than apreset depth value. More specifically, the multi-view image generator120 separates a foreground image and a rear ground image using the inputdepth information, and generates a multi-view foreground image and amulti-view rear ground image by processing the foreground image and therear ground image appropriately according to the type of image.

The foreground image and the rear ground image may be separated usingthe depth information. A method for separating the foreground image andthe rear ground image is described below.

In this exemplary embodiment, depth information of 8 bits is used. Eachpixel may have a depth value ranging from 0 to 255. In this case, areference depth value for separating the foreground and rear ground maybe set to 128. If depth information of a pixel is less than 128, thepixel belongs to a foreground image, and if depth information of a pixelis equal to or greater than 128, the pixel belongs to a rear groundimage. That is, the foreground image is a set of pixels having depthinformation which is less than 128, and the rear ground image is a setof pixels having depth information which is equal to or greater than128.

A preset depth value may be 128 which is the median value of depthvalues obtained by the depth information. A preset depth value forseparating a foreground image and a rear ground image is generally setto the median value, but it may be modified.

A method for separating a foreground image and a rear ground image willbe described below in greater detail with reference to FIG. 4. A methodfor generating a multi-view foreground image and a multi-view rearground image using the separated foreground image and rear ground imageis described below.

In the 3D display apparatus 100 according to the present exemplaryembodiment, if “n” view images are given corresponding to “n” views,generating multi-view images is not needed. However, in order tophotograph “n” view images, the “n” view images should be photographedby “N” photographing devices at the same time, and it is actuallydifficult to receive data of a high capacity.

Accordingly, “n” virtual view images are generated using a single-viewimage or view images under “N”.

Virtual multi-view images are generated using depth information of eachview and images of adjacent views.

For example, if 2 views of the images are used, a plurality of new viewsare made between the 2 views of the images. Subsequently, the 1^(st)view image and the 2^(nd) view image are sent to 3D space using theinput depth information, and a desired view is made.

Such an image processing technology is referred to as 3D warping. Ingeneral, a multi-view image which is generated using 2 view imagesrather than a single view image has less distortion.

In this manner, the multi-view image generator 120 may generate amulti-view foreground image and a multi-view rear ground image using theinput image and depth information of the image.

The multi-view image renderer 130 performs rendering by arranging themulti-view foreground image according to the first arrangement patternand by arranging the multi-view rear ground image according to thesecond arrangement pattern. The second arrangement pattern may be thesame as the first one according to an exemplary embodiment.

Arrangement patterns to reduce dead zones are described according toeach exemplary embodiment.

The first and second arrangement pattern according to the firstexemplary embodiment is described here.

If there are “N” multi-view foreground images and “N” multi-view rearground images, examples of the first arrangement pattern and secondarrangement pattern according to the first exemplary embodiment maydiffer according to whether “N” is an odd number or even number, andthus are described separately.

If “N” is an odd number (2K−1, K is a natural number), the firstarrangement pattern may be a repeat of a pattern where the 1^(st) viewto K^(th) view are arranged sequentially and then the K−1^(th) view to1^(st) view are arranged in reverse order. The second arrangementpattern may be a pattern obtained by shifting the first arrangementpattern by a predetermined number of views and thereby having a phasedifference.

For example, if there are a total of 9 views, the first arrangementpattern according to the first exemplary embodiment is a pattern wherethe 1^(st), 2^(nd), 3^(rd), 4^(th), 5^(th), 4^(th), 3^(rd), 2^(nd), and1^(st) view foreground images are arranged repeatedly, and the secondarrangement pattern is a pattern where 2^(nd), 3^(nd), 4^(th), 5^(th),4^(th), 3^(rd), 2^(nd), 1^(st), and 2^(nd) view rear ground images arearranged repeatedly.

On the other hand, if “N” is an even number (2K, K is a natural number),the first arrangement pattern may be a repeat of a pattern where the1^(st) view to K+1^(th) view are arranged sequentially and then theK^(th) view to 2^(nd) view are arranged in reverse order. The secondarrangement pattern may be a pattern obtained by shifting the firstarrangement pattern by a predetermined number of views and therebyhaving a phase difference.

For example, if there are a total of 8 views, the first arrangementpattern according to the first exemplary embodiment is a pattern wherethe 1^(st), 2^(nd), 3^(rd), 4^(th), 5, 4^(th), 3^(rd) and 2^(nd) viewforeground images are arranged repeatedly, and the second arrangementpattern is a pattern where 2^(nd), 3^(rd), 4^(th), 5^(th), 4^(th),3^(rd), 2^(nd), and 1^(st) view foreground images are arrangedrepeatedly.

The arrangement patterns according to the first exemplary embodimentwill be described below with reference to FIGS. 5 and 6.

The first and second arrangement pattern according to the secondexemplary embodiment is described here.

In the second exemplary embodiment, there are “N” multi-view foregroundimages and “N” multi-view rear ground images, as in the first exemplaryembodiment. However, unlike the first exemplary embodiment, examples ofthe first arrangement pattern and second arrangement pattern accordingto the second exemplary embodiment are the same regardless of whether“N” is an odd number or even number, and thus are not separatelydescribed.

The first arrangement pattern according to the second exemplaryembodiment may be a repeat of a pattern where the odd views of the1^(st) to N^(th) views are arranged sequentially and then the even viewsof the N^(th) to 1^(st) views are arranged in reverse order. The secondarrangement pattern may be a pattern obtained by shifting the firstarrangement pattern by a predetermined number of views and therebyhaving a phase difference.

For example, if there are a total of 9 views, the first arrangementpattern according to the second exemplary embodiment is a pattern wherethe 1^(st), 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th), 4^(th), and2^(nd) view foreground images are arranged repeatedly, and the secondarrangement pattern is a pattern where the 3^(rd), 5^(th), 7^(th),9^(th), 8^(th), 6^(th), 4^(th), 2^(nd), and 1^(st) view rear groundimages are arranged repeatedly.

On the other hand, if there are a total of 8 views, the firstarrangement pattern according to the second exemplary embodiment is apattern where the 1^(st), 3^(rd), 5^(th), 7^(th), 8^(th), 6^(th),4^(th), and 2^(nd) view foreground images are arranged repeatedly, andthe second arrangement pattern is a pattern where the 3^(rd), 5^(th),7^(th), 8^(th), 6^(th), 4^(th), 2^(nd), and 1^(st) view rear groundimages are arranged repeatedly.

The arrangement patterns according to the second exemplary embodimentwill be described below with reference to FIGS. 5 and 6.

The first and second arrangement pattern according to the thirdexemplary embodiment is described here.

In the third exemplary embodiment, there are “N” multi-view foregroundimages and “N” multi-view rear ground images, as in the first and secondexemplary embodiments. However, unlike the first and second exemplaryembodiments, the first arrangement pattern and second arrangementpattern according to the third exemplary embodiment are differentlydesigned.

The first arrangement pattern according to the third exemplaryembodiment may be a repeat of a pattern where the odd views of the1^(st) to N^(th) views are arranged sequentially and then the even viewsof the N^(th) to 1^(st) views are arranged in reverse order. If N is anodd number, the second arrangement pattern may be a repeat of a patternwhere the 1^(st) view is arranged first, the even views of the 2^(nd) toN^(th) views are arranged sequentially and then the odd views of theN^(th) to 2^(nd) views are arranged in reverse order. If N is an evennumber, the second arrangement pattern may be a repeat of a patternwhere the even views of the 1^(st) to N^(th) views are arrangedsequentially and then the odd views of the N^(th) to 1^(st) views arearranged in reverse order.

For example, if there are a total of 9 views, the first arrangementpattern according to the third exemplary embodiment is a pattern wherethe 1^(st), 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th), 4^(th), and2^(nd) view foreground images are arranged repeatedly, and the secondarrangement pattern is a pattern where the 1^(st), 2^(nd), 4^(th),6^(th), 8^(th), 9^(th), 7^(th), 5^(th), and 3^(rd) view rear groundimages are arranged repeatedly.

On the other hand, if there are a total of 8 views, the firstarrangement pattern according to the third exemplary embodiment is apattern where the 1^(st), 3^(rd), 5, 7^(th), 8^(th), 6^(th), 4^(th), and2^(nd) view foreground images are arranged repeatedly, and the secondarrangement pattern is a pattern where the 2^(nd), 4^(th), 6^(th),8^(th), 7^(th), 5^(th), 3^(rd), and 1^(st) view rear ground images arearranged repeatedly.

The arrangement patterns according to the third exemplary embodimentwill be described below with reference to FIGS. 9 and 10.

The multi-view foreground images and rear ground images are designedwith the arrangement patterns as described above. Accordingly, whereverthe viewer is positioned, a sudden change of image view may not occur.Consequently, dead zones are not generated.

In addition, even when multi-view images where foreground images andrear ground images are not separated are arranged and rendered as in thefirst arrangement patterns of the first, second and third exemplaryembodiments, a sudden change of image view may be reduced, therebyobtaining the effect as described above.

However, in multi-view images where foreground images and rear groundimages are not separated, a stereo section where views are sequentiallyarranged and a pseudo stereo section where views are arranged in reverseorder occur. Accordingly, impact may occur by switching between a stereosection and a pseudo stereo section according to a position of theviewer.

Therefore, the present exemplary embodiment uses a pattern where amulti-view foreground image and a multi-view rear ground image arearranged with a predetermined phase difference so that the impactgenerated by switching between a stereo section and a pseudo stereosection may be relieved. Further, a pseudo stereo section which makesthe viewing of the image by the viewer uncomfortable, may be reduced.

The display 140 outputs a rendered multi-view image. More specifically,the display 140 optically separates a multi-view image rendered by themulti-view image renderer 130 and displays the separated multi-viewimage. Methods for optically separating a multi-view image may useparallax barriers or lenticular lenses. The multi-view image separatedby the methods above is repeatedly separated and displayed in front ofthe 3D display apparatus 100 so that the viewer can watch a 3D image bybinocular disparity.

In view of the above, the 3D display apparatus 100 enables the viewer towatch a 3D image conveniently at any position using a multi-view imageprocessing method in order to reduce dead zones.

FIG. 4 illustrates an operation of the multi-view image generator 120according to an exemplary embodiment. More specifically, FIG. 4illustrates an operation of separating an image input by the image inputdevice 110 into a foreground image and a rear ground image using depthinformation.

The entire image includes a foreground image and a rear ground image. Asdescribed with reference to FIG. 1, a foreground image is a set ofpixels having depth information which is less than a preset depth value,and a rear ground image is a set of pixels having depth informationwhich is equal to or greater than the preset depth value.

For convenience of description, in the exemplary embodiment depthinformation is 8 bits and each pixel has a depth value ranging from 0 to255.

In the exemplary embodiment illustrated in FIG. 4, depth information ofpixel values indicating a shaded tree 410 is 70, depth information ofpixel values indicating a black tree 420 is 150, and depth informationof pixel values indicating a background 430 is 255. On the basis of thesetting of a reference depth value for separating the foreground andrear ground at 125, the shaded tree 410 having a depth value under 125is referred to as a foreground image, and the black tree 420 andbackground 430 having a depth value over 125 are referred to as a rearground image. As described with reference to FIG. 4, the multi-viewimage generator 120 separates a foreground image and a rear ground imageusing the preset reference value.

Patterns for arranging multi-view images using the multi-view imagerenderer 130 are described with reference to FIGS. 5 to 10. Here, amulti-view foreground image and a multi-view rear ground image each have9 views, but it may also be possible that a multi-view foreground imageand a multi-view rear ground image each have any number of othermulti-views, such as 8 views.

FIGS. 5 and 6 illustrate an arrangement pattern of multi-view imagesaccording to the first exemplary embodiment.

The first arrangement pattern according to the first exemplaryembodiment, i.e. an arrangement pattern of multi-view foreground imagesis a repeat of a pattern where the 1^(st) view to K^(th) view arearranged sequentially and then the K−1^(th) view to 1^(st) view arearranged in reverse order. The second arrangement pattern, i.e. anarrangement pattern of multi-view rear ground images is a patternobtained by shifting the first arrangement pattern by a predeterminednumber of views and thereby having a phase difference. The predeterminednumber of views are preferably 1 view, but may be modified according tothe setting of the user.

With reference to FIG. 5, multi-view foreground images and multi-viewrear ground images may be arranged as in Table 1 below, according to anoptical view.

TABLE 1 Optical view 1 2 3 4 5 6 7 8 9 1 2 3 Foreground 1 2 3 4 5 4 3 21 1 2 3 image view Rear ground 1 1 2 3 4 5 4 3 2 1 1 2 image view

As illustrated in FIG. 5, at the 5^(th) and 6^(th) optical views, theforeground image views are pseudo stereo, and the rear ground imageviews are stereo. The foreground image views switch from stereo topseudo stereo at the 5^(th) and 6^(th) optical views, and the rearground image views switch from stereo to pseudo stereo at the 6^(th) and7^(th) optical views. Since the foreground image views switch fromstereo to pseudo stereo first and then the rear ground image viewsswitch from stereo to pseudo stereo, impact generated by switching fromstereo to pseudo stereo may be relieved. In addition, the pseudo stereosection is less than in a multi-view image in which a foreground and arear ground are not separated so that a pseudo stereo section whichmakes the viewing of the image by the viewer awkward, may be reduced.

FIG. 6 illustrates operation of the 3D display apparatus 100 whichoptically separates and displays multi-view images rendered according tothe first exemplary embodiment. In FIG. 6, views of the foregroundimages and rear ground images arranged according to the first exemplaryembodiment are illustrated corresponding to each optical view. Forexample, if the viewer is positioned at the 5^(th) and 6^(th) opticalviews, the left eye of the viewer may watch the 5^(th) view foregroundimage and the 4^(th) view rear ground image, and the right eye may watchthe 4^(th) view foreground image and the 5^(th) view rear ground image.

FIGS. 7 and 8 illustrate an arrangement pattern of multi-view imagesaccording to the second exemplary embodiment.

The first arrangement pattern according to the second exemplaryembodiment, i.e. an arrangement pattern of multi-view foreground imagesis a repeat of a pattern where the odd views of the 1^(st) to N^(th)views are arranged sequentially and then the even views of the N^(th) to1^(st) views are arranged in reverse order. The second arrangementpattern, i.e. an arrangement pattern of multi-view rear ground images isa pattern obtained by shifting the first arrangement pattern by apredetermined number of views and thereby having a phase difference. Thepredetermined number of views are preferably 1 view, but may be modifiedaccording to a setting of the user.

With reference to FIG. 7, multi-view foreground images and multi-viewrear ground images may be arranged as in Table 2 below, according to anoptical view.

TABLE 2 Optical view 1 2 3 4 5 6 7 8 9 1 2 3 Foreground 1 3 5 7 9 8 6 42 1 3 5 image view Rear ground 2 1 3 5 7 9 8 6 4 2 1 3 image view

As illustrated in FIG. 7, at the 5^(th) and 6^(th) optical views, theforeground image views are pseudo stereo, and the rear ground imageviews are stereo. The foreground image views switch from stereo topseudo stereo at the 5^(th) and 6^(th) optical views, and the rearground image views switch from stereo to pseudo stereo at the 6^(th) and7^(th) optical views. Since the foreground image views switch fromstereo to pseudo stereo first and then the rear ground image viewsswitch from stereo to pseudo stereo, the impact generated by switchingfrom stereo to pseudo stereo may be relieved. In addition, the pseudostereo section is less than in a multi-view image in which a foregroundand a rear ground are not separated so that a pseudo stereo sectionwhich makes the image viewed by the viewer awkward, may be reduced.

FIG. 8 illustrates an operation of the 3D display apparatus 100 whichoptically separates and displays multi-view images rendered according tothe second exemplary embodiment. In FIG. 8, views of the foregroundimages and rear ground images arranged according to the second exemplaryembodiment are illustrated corresponding to each optical view. Forexample, if the viewer is positioned at the 5^(th) and 6^(th) opticalviews, the left eye of the viewer may watch the 9^(th) view foregroundimage and the 7^(th) view rear ground image, and the right eye may watchthe 8^(th) view foreground image and the 9^(th) view rear ground image.

FIGS. 9 and 10 illustrate an arrangement pattern of multi-view imagesaccording to the third exemplary embodiment.

The first arrangement pattern according to the third exemplaryembodiment, i.e. an arrangement pattern of multi-view foreground imagesis a repeat of a pattern where the odd views of the 1^(st) to N^(th)views are arranged sequentially and then the even views of the N^(th) to1^(st) views are arranged in reverse order. If N is an odd number, thesecond arrangement pattern, i.e. an arrangement pattern of multi-viewrear ground images is a repeat of a pattern where the 1^(st) view isarranged first, the even views of the 2^(nd) to N^(th) views arearranged sequentially, and then the odd views of the N^(th) to 2^(nd)views are arranged in reverse order. If N is an even number, the secondarrangement pattern is a repeat of a pattern where the even views of the1^(st) to N^(th) views are arranged sequentially and then the odd viewsof the N^(th) to 1^(st) views are arranged in reverse order.

With reference to FIG. 9, multi-view foreground images and multi-viewrear ground images may be arranged as in Table 3 below, according to anoptical view.

TABLE 3 Optical view 1 2 3 4 5 6 7 8 9 1 2 3 Foreground 1 3 5 7 9 8 6 42 1 3 5 image view Rear ground 1 2 4 6 8 9 7 5 3 1 2 4 image view

As illustrated in FIG. 9, at the 5^(th) and 6^(th) optical views, theforeground image views are pseudo stereo, and the rear ground imageviews are stereo. The foreground image views switch from stereo topseudo stereo at the 5^(th) and 6^(th) optical views, and the rearground image views switch from stereo to pseudo stereo at the 6^(th) and7^(th) optical views. Since the foreground image views switch fromstereo to pseudo stereo first and then the rear ground image viewsswitch from stereo to pseudo stereo, the impact generated by switchingfrom stereo to pseudo stereo may be relieved. In addition, the pseudostereo section is less than in a multi-view image in which a foregroundand a rear ground are not separated so that a pseudo stereo sectionwhich makes the viewer awkward may be reduced.

FIG. 10 illustrates an operation of the 3D display apparatus 100 whichoptically separates and displays multi-view images rendered according tothe third exemplary embodiment. In FIG. 10, views of the foregroundimages and rear ground images arranged according to the third exemplaryembodiment are illustrated corresponding to each optical view. Forexample, if the viewer is positioned at the 5^(th) and 6^(th) opticalviews, the left eye of the viewer may watch the 9^(th) view foregroundimage and the 8^(th) view rear ground image, and the right eye may watchthe 8^(th) view foreground image and the 9^(th) view rear ground image.

FIG. 11 illustrates operation of the multi-view image renderer 130according to an exemplary embodiment. More specifically, a method forarranging multi-view images according to a preset arrangement patternand rendering the multi-view images as a single frame to be displayed isdescribed here. In this exemplary embodiment, a multi-view foregroundimage and a multi-view rear ground image each have 8 views.

With reference to FIG. 11, from among a foreground area 1110 and a rearground area 1120 of an image separated by the multi-view image generator120, in the foreground area 1110, a multi-view foreground image isrendered using only pixels arranged according to a preset arrangementpattern, and in the rear ground area 1120, a multi-view rear groundimage is rendered using only pixels arranged according to a presetarrangement pattern.

In FIG. 11, the pixel arrangement drawing for each area adopts anarrangement pattern according to the third exemplary embodiment, thefigure written in each pixel indicates the number of each view, “f”indicates a foreground, and “b” indicates a rear ground.

As described above, the multi-view image renderer 130 renders amulti-view foreground image and a multi-view rear ground image which arearranged according to a preset arrangement pattern using a single frame,and provides the display 140 with the rendered multi-view image.

FIG. 12 is a flow chart illustrating a method for processing an imageusing the 3D display apparatus 100 according to an exemplary embodiment.

With reference to FIG. 12, in operation S1210, the 3D display apparatus100 receives an image and depth information of the image.

In operation S1220, the 3D display apparatus 100 generates a multi-viewforeground image having depth information less than a preset depthvalue, and a multi-view rear ground image having depth informationhigher than the preset depth value based on the received image and thereceived depth information of the image.

A method for generating the multi-view foreground image and themulti-view rear ground image has been described with reference to FIG.1, so detailed description is not repeated.

In operation S1230, the 3D display apparatus 100 performs rendering byarranging the multi-view foreground image according to the firstarrangement pattern and by arranging the multi-view rear ground imageaccording to the second arrangement pattern.

The multi-view foreground image and the multi-view rear ground imageeach may have “N” views.

In the first exemplary embodiment, if “N” is an odd number (2K−1, K is anatural number), the first arrangement pattern may be a repeat of apattern where the 1^(st) view to K^(th) view are arranged sequentiallyand then the K−1^(th) view to 1st view are arranged in reverse order. If“N” is an even number (2K, K is a natural number), the first arrangementpattern may be a repeat of a pattern where the 1^(st) view to K+1^(th)view are arranged sequentially and then the K^(th) view to 2^(nd) vieware arranged in reverse order. The second arrangement pattern may be apattern obtained by shifting the first arrangement pattern by apredetermined number of views and thereby having a phase difference.

In the second exemplary embodiment, the first arrangement pattern may bea repeat of a pattern where the odd views of the 1^(st) to N^(th) viewsare arranged sequentially and then the even views of the N^(th) to1^(st) views are arranged in reverse order. The second arrangementpattern may be a pattern obtained by shifting the first arrangementpattern by a predetermined number of views and thereby having a phasedifference.

In the third exemplary embodiment, the first arrangement pattern may bea repeat of a pattern where the odd views of the 1^(st) to N^(th) viewsare arranged sequentially and then the even views of the N^(th) to1^(st) views are arranged in reverse order. If N is an odd number, thesecond arrangement pattern may be a repeat of a pattern where the 1^(st)view is arranged first, the even views of the 2^(nd) to N^(th) views arearranged sequentially and then the odd views of the N^(th) to 2^(nd)views are arranged in reverse order. Or, if N is an even number, thesecond arrangement pattern may be a repeat of a pattern where the evenviews of the 1^(st) to N^(th) views are arranged sequentially and thenthe odd views of the N^(th) to 1^(st) views are arranged in reverseorder.

In operation S1240, after rendering is completed, the 3D displayapparatus 100 outputs the rendered multi-view image.

The image processing method of the 3D display apparatus 100 described inFIG. 12 may be performed by the 3D display apparatus 100 having aconfiguration of FIG. 3, and may also be performed by any other 3Ddisplay apparatuses having a different configuration.

In view of the above, the 3D display apparatus 100 according to theexemplary embodiment processes an image using an image processing methodto reduce dead zones, thereby allowing the viewer to watch a 3D imagecomfortably at any position.

Furthermore, the 3D display apparatus 100 according to the exemplaryembodiment processes an image using a pattern where a multi-viewforeground image and a multi-view rear ground image are arranged with apredetermined phase difference so that the impact generated by switchingbetween a stereo section and a pseudo stereo section may be relieved. Inaddition, a pseudo stereo section which makes the viewing of the imageby the viewer uncomfortable, may be reduced.

These image processing methods according to the diverse exemplaryembodiments are programmed and stored in diverse types of storage media,and thus may be implemented by diverse types of electronic devicesexecuting the storage media.

In addition, the image processing method as described above may beimplemented in a program including an algorithm which can be executed bya computer. The program may be stored and provided in a non-transitorycomputer readable medium.

A non-transitory computer readable medium is a medium which does notstore data temporarily such as a register, cash, and memory but storesdata semi-permanently and is readable by a device. More specifically,the aforementioned diverse applications or programs may be stored andprovided in a non-transitory computer readable medium such as a compactdisk (CD), digital video disk (DVD), hard disk, Blu-ray disk, universalserial bus (USB), memory card, and read-only memory (ROM).

The foregoing exemplary embodiments are merely exemplary and are not tobe construed as limiting the inventive concept. The present teaching canbe readily applied to other types of apparatuses. Also, the descriptionof the exemplary embodiments is intended to be illustrative, and not tolimit the scope of the claims, and many alternatives, modifications, andvariations will be apparent to those skilled in the art.

What is claimed is:
 1. A three-dimensional (3D) display apparatuscomprising: an image input device which receives an image and depthinformation of the image; a multi-view image generator which generates amulti-view foreground image which has depth information which is lessthan a preset depth value, and a multi-view rear ground image which hasdepth information which is equal to or greater than the preset depthvalue, based on the received image and the received depth information ofthe image; a multi-view image renderer which performs rendering byarranging the multi-view foreground image according to a firstarrangement pattern and arranging the multi-view rear ground imageaccording to a second arrangement pattern different from the firstarrangement pattern; and a display which outputs the rendered multi-viewimage, wherein the received image is one of a single-view image, astereoscopic image, and a multi-view image.
 2. The 3D display apparatusas claimed in claim 1, wherein the multi-view foreground image and themulti-view rear ground image each have “N” views, wherein when “N” is anodd number (2K−1, K is a natural number), the first arrangement patternis a repeat of a pattern where a 1^(st) view to a K^(th) view arearranged sequentially and then a K−1^(th) view to the 1^(st) view arearranged in reverse order, when “N” is an even number (2K, K is anatural number), the first arrangement pattern is a repeat of a patternwhere the 1^(st) view to a K+1^(th) view are arranged sequentially andthen the K^(th) view to a 2^(nd) view are arranged in reverse order, andthe second arrangement pattern is a pattern obtained by shifting thefirst arrangement pattern by a predetermined number of views to have aphase difference.
 3. The 3D display apparatus as claimed in claim 2,wherein the multi-view foreground image and the multi-view rear groundimage each have 9 views, the first arrangement pattern is a patternwhere 1^(st), 2^(nd), 3^(rd), 4^(th), 5^(th), 4^(th), 3^(rd), 2^(nd),and 1^(st) view foreground images are arranged repeatedly, and thesecond arrangement pattern is a pattern where 2^(nd), 3^(rd), 4^(th),5^(th), 4^(th), 3^(rd), 2^(nd), 1^(st) and 2^(nd) view rear groundimages are arranged repeatedly.
 4. The 3D display apparatus as claimedin claim 1, wherein the multi-view foreground image and the multi-viewrear ground image each have “N” views, the first arrangement pattern isa repeat of a pattern where odd views of a 1^(st) to N^(th) views arearranged sequentially and then even views of the N^(th) to the 1^(st)views are arranged in reverse order, and the second arrangement patternis a pattern obtained by shifting the first arrangement pattern by apredetermined number of views to have a phase difference.
 5. The 3Ddisplay apparatus as claimed in claim 4, wherein the multi-viewforeground image and the multi-view rear ground image each have 9 views,the first arrangement pattern is a pattern where 1^(st), 3^(rd), 5^(th),7^(th), 9^(th), 8^(th), 6^(th), 4^(th), and 2^(nd) view foregroundimages are arranged repeatedly, and the second arrangement pattern is apattern where 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th), 4^(th)2^(nd), and 1^(st) view foreground images are arranged repeatedly. 6.The 3D display apparatus as claimed in claim 1, wherein the multi-viewforeground image and the multi-view rear ground image each have “N”views, the first arrangement pattern is a repeat of a pattern where oddviews of a 1^(st) to N^(th) views are arranged sequentially and theneven views of the N^(th) to the 1^(st) views are arranged in reverseorder, and when N is an odd number, the second arrangement pattern is arepeat of a pattern where the 1^(st) view is arranged first, even viewsof 2^(nd) to the N^(th) views are arranged sequentially and then oddviews of the N^(th) to the 2^(nd) views are arranged in reverse order,or when N is an even number, the second arrangement pattern is a repeatof a pattern where even views of the 1^(st) to the N^(th) views arearranged sequentially and then odd views of N^(th) to 1^(st) views arearranged in reverse order.
 7. The 3D display apparatus as claimed inclaim 6, wherein the multi-view foreground image and the multi-view rearground image each have 9 views, the first arrangement pattern is apattern where 1^(st), 3^(rd), 5^(th), 7^(th), 9^(th), 8^(th), 6^(th),4^(th), and 2^(nd) view foreground images are arranged repeatedly, andthe second arrangement pattern is a pattern where 1^(st), 2^(nd),4^(th), 6^(th), 8^(th), 9^(th), 7^(th), 5^(th), and 3^(rd) view rearground images are arranged repeatedly.
 8. An image processing method ofa three-dimensional (3D) display apparatus, the method comprising:receiving an image and depth information of the image; generating amulti-view foreground image having depth information which is less thana preset depth value, and a multi-view rear ground image having depthinformation which is equal to or greater than the preset depth value,using the received image and the received depth information of theimage; performing rendering by arranging the multi-view foreground imageaccording to a first arrangement pattern and arranging the multi-viewrear ground image according to a second arrangement pattern differentfrom the first arrangement pattern; and outputting the renderedmulti-view image, wherein the received image is one of a single-viewimage, a stereoscopic image, and a multi-view image.
 9. The imageprocessing method as claimed in claim 8, wherein the multi-viewforeground image and the multi-view rear ground image each have “N”views, when “N” is an odd number (2K−1, K is a natural number), thefirst arrangement pattern is a repeat of a pattern where a 1^(st) viewto a K^(th) view are arranged sequentially and then a K−1^(th) view tothe 1^(st) view are arranged in reverse order, when “N” is an evennumber (2K, K is a natural number), the first arrangement pattern is arepeat of a pattern where the 1^(st) view to a K+1^(th) view arearranged sequentially and then the K^(th) view to a 2^(nd) view arearranged in reverse order, and the second arrangement pattern is apattern obtained by shifting the first arrangement pattern by apredetermined number of views to have a phase difference.
 10. The imageprocessing method as claimed in claim 8, wherein the multi-viewforeground image and the multi-view rear ground image each have “N”views, the first arrangement pattern is a repeat of a pattern where oddviews of a 1^(st) to N^(th) views are arranged sequentially and theneven views of the N^(th) to the 1^(st) views are arranged in reverseorder, and the second arrangement pattern is a pattern obtained byshifting the first arrangement pattern by a predetermined number ofviews to have a phase difference.
 11. The image processing method asclaimed in claim 8, wherein the multi-view foreground image and themulti-view rear ground image each have “N” views, the first arrangementpattern is a repeat of a pattern where odd views of a 1^(st) to N^(th)views are arranged sequentially and then even views of the N^(th) to the1^(st) views are arranged in reverse order, and when N is an odd number,the second arrangement pattern is a repeat of a pattern where the 1^(st)view is arranged first, even views of a 2^(nd) to the N^(th) views arearranged sequentially and then odd views of the N^(th) to the 2^(nd)views are arranged in reverse order, or when N is an even number, thesecond arrangement pattern is a repeat of a pattern where even views ofthe 1^(st) to the N^(th) views are arranged sequentially and then oddviews of the N^(th) to the 1^(st) views are arranged in reverse order.12. A non-transitory computer readable medium which includes a programto execute an image processing method of a three-dimensional (3D)display apparatus, wherein the image processing method comprises:receiving an image and depth information of the image; generating amulti-view foreground image having depth information which is less thana preset depth value, and a multi-view rear ground image having depthinformation which is equal to or greater than the preset depth value,using the received image and the received depth information of theimage; performing rendering by arranging the multi-view foreground imageaccording to a first arrangement pattern and arranging the multi-viewrear ground image according to a second arrangement pattern differentfrom the first arrangement pattern; and outputting the renderedmulti-view image, wherein the received image is one of a single-viewimage, a stereoscopic image, and a multi-view image.
 13. Athree-dimensional (3D) display apparatus comprising: an image inputdevice, which receives an image and depth information of the image; amulti-view image generator, which generates a multi-view foregroundimage and a multi view rear ground image based on the received image, amulti-view image renderer, which renders the image by arranging themulti-view foreground image based on a first arrangement pattern, andarranging the multi-view rear ground image by a second arrangementpattern different from the first arrangement pattern, and a displaywhich outputs the rendered multi-view image, wherein the received imageis one of a single-view image, a stereoscopic image, and a multi-viewimage.
 14. The 3D apparatus of claim 13, wherein the multi-viewforeground image has a depth which is less than a preset depth value,and the multi view rear ground image has a depth which is equal to orgreater than the preset depth value.